Tutorial
This tutorial shows the functionalities of Extremes.jl. They are illustrated by reproducing some of the results shown by Coles (2001) in An Introduction to Statistical Modeling of Extreme Values.
Before executing this tutorial, make sure to have installed the following packages:
- Extremes.jl (of course)
- DataFrames.jl (for using the DataFrame type)
- Distributions.jl (for using probability distribution objects)
- Gadfly.jl (for plotting)
and import them using the following command:
julia> using Extremes, DataFrames, Distributions, GadflyModel for stationary block maxima
Port Pirie example
This section concerns the annual maximum sea-levels recorded at Port Pirie, South Australia, from 1923 to 1987. This dataset were studied by Coles (2001) in Chapter 3.
Load the data
Loading the annual maximum sea-levels at Port Pirie:
data = load("portpirie")
first(data,5)| Year | SeaLevel | |
|---|---|---|
| Int64 | Float64 | |
| 1 | 1923 | 4.03 |
| 2 | 1924 | 3.83 |
| 3 | 1925 | 3.65 |
| 4 | 1926 | 3.88 |
| 5 | 1927 | 4.01 |
Plotting the data using the Gadfly package:
plot(data, x=:Year, y=:SeaLevel, Geom.line)
GEV parameters estimation
In this example, the Generalized Extreme Value (GEV) distribution is fitted by maximum likelihood to the annual maximum sea-levels at Port-Pirie.
The data have been loaded in a DataFrame. The function gevfit can be called directly using the dataframe as the first argument and the data column symbol as the second argument as follows:
julia> fm = gevfit(data, :SeaLevel)
MaximumLikelihoodEVA
model :
BlockMaxima
data : Array{Float64,1}[65]
location : μ ~ 1
logscale : ϕ ~ 1
shape : ξ ~ 1
θ̂ : [3.874750223091266, -1.6192723640210762, -0.05010719929448139]The function gevfit returns a MaximumLikelihoodEVA object which contains:
- the structure name indicating in particular the estimation method (maximum likelihood in this example);
- the statistical model (the stationary block maxima model in this example);
- the location, log-scale and shape parameter estimates respectively in the vector $ θ̂ $.
The function returns the estimates of the log-scale parameter $\phi = \log \sigma$.
Diagnostics plots
TODOReturn level estimation
T-year return level estimate can be obtained using the function returnlevel on a fittedEVA object. The first argument is the fitted model, the second is the return period in years and the last one is the confidence level for computing the confidence interval.
For example, the 100-year return level for the Port Pirie blockmaxima model and the corresponding 95% confidence interval can be obtained with this commands:
julia> r = returnlevel(fm, 100, .95)
ReturnLevel(MaximumLikelihoodEVA
model :
BlockMaxima
data : Array{Float64,1}[65]
location : μ ~ 1
logscale : ϕ ~ 1
shape : ξ ~ 1
θ̂ : [3.874750223091266, -1.6192723640210762, -0.05010719929448139]
, 100, [4.688403360432851], [[4.377121171613511, 4.999685549252191]])where the value can be accessed with
julia> r.value
1-element Array{Float64,1}:
4.688403360432851and where the corresponding confidence interval can be accessed with
julia> r.cint
1-element Array{Array{Float64,1},1}:
[4.377121171613511, 4.999685549252191]In this example of a stationary model, the function returns a unit dimension vector for the return level and a vector containing only one vector for the confidence interval. The reason is that the function always returns the same type in the stationary and non-stationary case. The function is therefore type-stable allowing better performance of code execution.
To get the scalar return level in the stationary case, the following command can be used:
julia> r.value[]
4.688403360432851To get the scalar confidence interval in the stationary case, the following command can be used:
julia> r.cint[]
2-element Array{Float64,1}:
4.377121171613511
4.999685549252191Probability weighted moments estimation
Probability weighted moments estimation of the GEV parameters can also be performed by using the gevfitpwm function. All the methods also apply to the pwmEVA object.
julia> fm = gevfitpwm(data[:,:SeaLevel])
pwmEVA
model :
BlockMaxima
data : Array{Float64,1}[65]
location : μ ~ 1
logscale : ϕ ~ 1
shape : ξ ~ 1
θ̂ : [3.8731723562720766, -1.5932320395836068, -0.051477125862911276]Bayesian estimation
Bayesian estimation of the GEV parameters can also be performed by using the gevfitbayes function. All the methods also apply to the BayesianEVA object.
julia> fm = gevfitbayes(data[:,:SeaLevel])
Progress: 0%| | ETA: 0:20:18[K
Progress: 12%|████▊ | ETA: 0:00:04[K
Progress: 18%|███████▍ | ETA: 0:00:03[K
Progress: 23%|█████████▌ | ETA: 0:00:02[K
Progress: 28%|███████████▍ | ETA: 0:00:02[K
Progress: 34%|██████████████▏ | ETA: 0:00:02[K
Progress: 40%|████████████████▍ | ETA: 0:00:02[K
Progress: 47%|███████████████████▏ | ETA: 0:00:01[K
Progress: 53%|█████████████████████▋ | ETA: 0:00:01[K
Progress: 58%|███████████████████████▋ | ETA: 0:00:01[K
Progress: 64%|██████████████████████████▍ | ETA: 0:00:01[K
Progress: 70%|████████████████████████████▊ | ETA: 0:00:01[K
Progress: 77%|███████████████████████████████▌ | ETA: 0:00:01[K
Progress: 83%|█████████████████████████████████▉ | ETA: 0:00:00[K
Progress: 88%|███████████████████████████████████▉ | ETA: 0:00:00[K
Progress: 94%|██████████████████████████████████████▊ | ETA: 0:00:00[K
Progress: 100%|█████████████████████████████████████████| Time: 0:00:02[K
BayesianEVA
model :
BlockMaxima
data : Array{Float64,1}[65]
location : μ ~ 1
logscale : ϕ ~ 1
shape : ξ ~ 1
sim : Mamba.ChainsModel for stationary threshold exceedances
The data of this section come from Chapter 4 of Coles (2001) and correspond to the daily rainfall accumulations at a location in south-west England from 1914 to 1962.
Load the data
Loading the daily rainfall at a location in South-England:
data = load("rain")
x = collect(Date(1914,1,1):Day(1):Date(1961,12,30))
data[!,:Date] = x
select!(data, [:Date, :Rainfall])
first(data,5)| Date | Rainfall | |
|---|---|---|
| Date… | Float64 | |
| 1 | 1914-01-01 | 0.0 |
| 2 | 1914-01-02 | 2.3 |
| 3 | 1914-01-03 | 1.3 |
| 4 | 1914-01-04 | 6.9 |
| 5 | 1914-01-05 | 4.6 |
Plotting the data using the Gadfly package:
plot(data, x=:Date, y=:Rainfall, Geom.point, Theme(discrete_highlight_color=c->nothing))
Threshold selection
TODO
GPD parameters estimation
Let's first identify the threshold exceedances:
threshold = 30.0
df = filter(row -> row.Rainfall > threshold, data)
first(df, 5)| Date | Rainfall | |
|---|---|---|
| Date… | Float64 | |
| 1 | 1914-02-07 | 31.8 |
| 2 | 1914-03-08 | 32.5 |
| 3 | 1914-12-17 | 31.8 |
| 4 | 1914-12-30 | 44.5 |
| 5 | 1915-02-13 | 30.5 |
Get the exceedances above the threshold:
df[!,:Rainfall] = df[!,:Rainfall] .- threshold
rename!(df, :Rainfall => :Exceedance)
first(df, 5)| Date | Exceedance | |
|---|---|---|
| Date… | Float64 | |
| 1 | 1914-02-07 | 1.8 |
| 2 | 1914-03-08 | 2.5 |
| 3 | 1914-12-17 | 1.8 |
| 4 | 1914-12-30 | 14.5 |
| 5 | 1915-02-13 | 0.5 |
Generalized Pareto parameter estimation by maximum likelihood:
julia> fm = gpfit(df, :Exceedance)
MaximumLikelihoodEVA
model :
ThresholdExceedance
data : Array{Float64,1}[152]
logscale : ϕ ~ 1
shape : ξ ~ 1
θ̂ : [2.006896498380506, 0.1844926991237574]The function returns the estimates of the log-scale parameter $\phi = \log \sigma$.
Return level estimation
With the ThresholdExceedance structure, the returnlevel function requires several arguments to calculate the T-year return level:
- the threshold value;
- the number of total observation (below and above the threshold);
- the number of observations per year;
- the return period T;
- the confidence level for computing the confidence interval.
The function uses the Peaks-Over-Threshold model definition (Coles, 2001, Chapter 4) for computing the T-year return level.
For the rainfall example, the 100-year return level can be estimated as follows:
julia> r = returnlevel(fm, threshold, size(data,1), 365, 100, .95)
ReturnLevel(MaximumLikelihoodEVA
model :
ThresholdExceedance
data : Array{Float64,1}[152]
logscale : ϕ ~ 1
shape : ξ ~ 1
θ̂ : [2.006896498380506, 0.1844926991237574]
, 100, [106.32558691303024], [[65.48163774428642, 147.16953608177405]])where the value can be accessed with
julia> r.value
1-element Array{Float64,1}:
106.32558691303024and where the corresponding confidence interval can be accessed with
julia> r.cint
1-element Array{Array{Float64,1},1}:
[65.48163774428642, 147.16953608177405]In this example of a stationary model, the function returns a unit dimension vector for the return level and a vector containing only one vector for the confidence interval. The reason is that the function always returns the same type in the stationary and non-stationary case. The function is therefore type-stable allowing better performance of code execution.
To get the scalar return level in the stationary case, the following command can be used:
julia> r.value[]
106.32558691303024To get the scalar confidence interval in the stationary case, the following command can be used:
julia> r.cint[]
2-element Array{Float64,1}:
65.48163774428642
147.16953608177405Probability weighted moments estimation
Probability weighted moments estimation of the GEV parameters can also be performed by using the gevfitpwm function. All the methods also apply to the pwmEVA object.
julia> fm = gpfitpwm(df, :Exceedance)
ERROR: MethodError: no method matching gpfitpwm(::DataFrame, ::Symbol)Bayesian estimation
Bayesian estimation of the GEV parameters can also be performed by using the gevfitbayes function. All the methods also apply to the `BayesianEVA object.
julia> fm = gpfitbayes(df, :Exceedance)
Progress: 11%|████▍ | ETA: 0:00:01[K
Progress: 22%|████████▉ | ETA: 0:00:01[K
Progress: 32%|█████████████▏ | ETA: 0:00:01[K
Progress: 44%|██████████████████▏ | ETA: 0:00:01[K
Progress: 55%|██████████████████████▌ | ETA: 0:00:00[K
Progress: 67%|███████████████████████████▍ | ETA: 0:00:00[K
Progress: 78%|████████████████████████████████ | ETA: 0:00:00[K
Progress: 90%|████████████████████████████████████▊ | ETA: 0:00:00[K
Progress: 100%|█████████████████████████████████████████| Time: 0:00:00[K
BayesianEVA
model :
ThresholdExceedance
data : Array{Float64,1}[152]
logscale : ϕ ~ 1
shape : ξ ~ 1
sim : Mamba.ChainsModel for dependent data
Coles(2001, Chapter 5)
Model for non-stationary data
Coles(2001, Chapter 6)